Restricted Eigenvalue from Stable Rank with Applications to Sparse Linear Regression

High-dimensional sparse linear regression is a basic problem in machine learning and statistics. Consider a linear model y = Xθ + w, where y ∈ R is the vector of observations, X ∈ R is the covariate matrix with ith row representing the covariates for the ith observation, and w ∈ R is an unknown noise vector. In many applications, the linear regression model is high-dimensional in nature, meaning that the number of observations n may be substantially smaller than the number of covariates d. In these cases, it is common to assume that θ is sparse, and the goal in sparse linear regression is to estimate this sparse θ, given (X,y). In this paper, we study a variant of the traditional sparse linear regression problem where each of the n covariate vectors in R are individually projected by a random linear transformation to R with m ≪ d. Such transformations are commonly applied in practice for computational savings in resources such as storage space, transmission bandwidth, and processing time. Our main result shows that one can estimate θ with a low l2-error, even with access to only these projected covariate vectors, under some mild assumptions on the problem instance. Our approach is based on solving a variant of the popular Lasso optimization problem. While the conditions (such as the restricted eigenvalue condition on X) for success of a Lasso formulation in estimating θ are well-understood, we investigate conditions under which this variant of Lasso estimates θ. The main technical ingredient of our result, a bound on the restricted eigenvalue on certain projections of a deterministic matrix satisfying a stable rank condition, could be of interest beyond sparse regression.